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Abstract. In this paper, we obtain a subconvexity result for the Rankin-Selberg L-function 
in both levels. The new feature in this result is applying an amplification method of Duke- 
Friedlander-Iwaniec to a double Petersson-Kuznetsov trace formula. As the trace formula 
' ranges over all the GL2 spectrum, this subconvexity result is unconditional of which Hecke 

04 ' eigenforms are chosen in the Rankin-Selberg L-function. 



1. Introduction 



^ ' We apply techniques gained from a beyond endoscopy approach to the Rankin-Selberg 
^ . L-function in [H] to understanding subconvexity. In [H], we studied an average of Rankin- 
Selberg L-functions L(/ x h,s). More specifically, for a smooth compactly supported test 
function g, we investigated 



n h f 



>■ ' as X gets large, with a„(/), an{g) being weighted Fourier coefficients associated to holomor- 
0\ phic or Maass Hecke eigenforms /, h. On applying Mellin inversion to this formula, we are 
^ studying averages of Rankin-Selberg L-functions. Understanding this sum is reminiscent to 
1: understanding an approximate functional equation for the Rankin-Selberg L-function. How- 
ever the important difference, which is explained in more detail below, is that the parameter 
^ ' X above has no apparent connection to the essential data (level, weight,..) of the L-function, 
^ while in the approximate function equation the parameter(s) which one investigates in sub- 
convexity problems are intrinsically associated to X. For example, the sum 



h f 



over forms /, h of level p, q, respectively, is roughly the approximate functional equation for 
the Rankin-Selberg L-function. In an approach to breaking convexity in both levels, one 
would naively consider this sum as p, g — 00. 

Given this connection of beyond endoscopy and the approximate functional equation, we 
ask if we can learn anything more about subconvexity for the Rankin-Selberg L-function. 
The clear connection of the two above sums hints we should study subconvexity as the two 
levels of the associated automorphic forms of the L-function go to infinity. 

This question of subconvexity for the Rankin-Selberg L-function in the level aspect has 
been studied in |DFI] . |KMV] .and |MV] . However, in these cases, one of the levels remains 
fixed as the other goes to infinity. Recently in |HM] the first case where both levels vary was 
announced. We state their result. Let M be a positive square-free integer and P a prime 
with (M, P) = 1, P ~ and < 77 < ^. Then for two holomorphic newforms /, h we have 
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the bound 

L{f X h,l/2) < (MP)i/2+e 



1 1 



(MP) 2(i+'7) (^Mp-^W(TTWj 

We make no such restrictions in this paper and associate the subconvexity of the Rankin- 
Selberg L-function in both levels to subconvexity bounds of Dirichlet L-functions. Our result 
is: 

Theorem 1.1. Let f,h be holomorphic or Maass Hecke eigenforms associated to GL2 au- 
tomorphic representations of prime level p, q, with p ^ q with nontrivial central characters 
Xyi^, respectively. Then for any e > and a fixed but arbitrarily small 6 > as p,q ^ 00, 
we have 

Lifxh,l/2) « ipqy(^ipq)f~'/^ + ipq)'/'+i+p'/'+'^q"^-'/^+q'^^^^^ 

Here 6 > comes from a (sub) convexity bound for the Dirichlet L-functions L{x, 1/2) ^ p^^*^ 
and L{^, 1/2) < 

Remark. The convexity bound for the Rankin- Selberg L-function in the level aspect is 

L(/x/^,l/2)«(M)^+^ 

Corollary 1.2. If x md if) are real quadratic characters, then we can apply Theorem li.il 
with d = \ using the result of [Clj to get the bound 

L{f X h,l/2) < (pg)^+^ 
// one of the characters is not real then we can apply the bound 6 = 0/ |B] to get 

27 

i:(/x/^,l/2)«(M)^+^ 

2. Details of Paper 

Inspired by [DFIj and |KMV] . we apply the amplification method. Ignoring convergence 
issues for now, we study 

(2.1) E \LU^hM2)\\ 

where xi are arbitrary complex coefficients which we optimize to "amplify" a single Rankin- 
Selberg L-function. Using an approximate functional equation and then a double Kuznetsov 
trace formula similar to [H], we bound (12. ip in terms of a function F of p, g, and L. In order 
to bound just a single L-function, we need positivity to get to an estimate 

l5^xA/(0r E \L{f^hM2)\^<t:F{p,q,xuL). 

1<L h&H{q,ip) 

Then choosing the xi to make | X]/<l^'-^/(OP large as possible ensures that 

F{p,q,xi,L) \ 



L(/x/i,l/2)« 



Ez<l3;zA/(/)| J 



is as small as possible. 

To the details of actually getting the bound F{p,q,xi, L) after applying the double 
Kuznetsov formula, we open up the Kloosterman sums and reorganize the Ci,C2-sums and 
associated exponential sums to get terms more amenable to estimation similar to jH] . 
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One then has four sums needing estimation. Two come from the geometric sides of the 
trace formulas (labelled Ci,C2), and the two sums from opening up \L{f x h,s)\'^ with the 
approximate functional equation. Performing Poisson summation and Mellin inversion at 
various steps reduces the estimation to GLi Dirichlet L-functions. 

Another feature in the calculation is that one sees a shadow of the Selberg trace formula. 
We motivate this connection with a short heuristic. First, we denote by H{p, x) the auto- 
morphic forms with level p and central character x- Ignoring test functions and convergence 
issues, the second moment calculation essential to this paper is: 



(2.2) J2 \Lil/2Jxh)\ 

f&H{p,l),heH{q,l) 



2 



Yl Yl 7 — KTH9i—)9'i—)af{n)af{m)ah{n)ah{m) 

QJpq git) \ 1 .,m 



^2 



™->i \ Fe-f/((p,<?),i) 



The second approximation is just using classical Rankin- Selberg theory on the n-sum, or 
one can also apply an analogous argument of [H]. Another application of Rankin-Selberg 
theory to the m-sum gives as a main term for (12. 2p : 



(2.3) ^-^g{l/2)g'{l/2) I i?e.,=iL(l, F x F) 



with 'g denoting the Mellin transform of g. But this term ReSs=iL[l, F x F) cancels the or- 
thonormalization built into the Kuznetsov trace formula. This final formula is an unweighted 
trace formula, or a Selberg trace formula. Though in this paper we take nontrivial central 
characters x mod {p),ip mod (g), something similar to this phenomenon is still seen. For- 
tunately recognizing the geometric side of the Selberg trace formula, in this fashion, has 
been encountered and studied by Rudnick in his thesis |R]. There also seems to be some 
connection of interactions of the geometric sides of the Kuznetsov and Selberg trace formula 
in the analysis in |SY] . which counts the number of closed geodesies up to a large parameter 
X. 

Next, though we explicitly avoided Bessel functions in our initial Kuznetsov trace 
formulas-in estimating the archimedean integrals that occur in our estimation-we do see 
Bessel functions. However, they display themselves "late" in the calculation, and we use 
standard asymptotics to deal with them. We say "late" in the sense that one could approach 
the above kind of subconvexity problem in Theorem 11.11 using one Kuznetsov trace formula 
(for the /-sum) and one Voronoi summation for a specific automorphic form h, very similar 
to |KMV] . In the early stages of such a calculation (specifically after Voronoi summation), 
one encounters difficult arithmetic and Bessel function analysis. Another difficulty is that 
one would like to understand subconvexity in both levels for all GL2 forms, so one would 
need a different Voronoi summation for each type of automorphic form h (holomorphic, 
Maass, and Eisenstein). Doing a double Kuznetsov trace formula simplifies this analysis. In 
another paper [HI] , we prove that the Kuznetsov (Petersson) trace formula implies Voronoi 
summation, so no information is lost in the approach of the current paper. 
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Finally, in order to ensure positivity of 
(2.4) Yl h{V,tf)\J2^iXM' E HWMmi/2J X h)\\ 

-where now we include the test functions that ensure convergence (as compared to fl2.1l) )-in 
section [TO]we discuss how we can choose V, W such that the test functions h{W,th), h(y,tf) 
are "large" for automorphic forms with eigenvalues (or weights) {tfo,tho} and arbitrarily 
small at any other spectral values. 



2.1. The Kloosterman-Kloosterman term. When we apply the trace formula to both 
spectral sums, we encounter four different terms: the delta-delta term, the delta-Kloosterman 
term, Kloosterman-delta term, and the Kloosterman-Kloosterman term. The first three are 
relatively straightforward to deal with. However, the Kloosterman-Kloosterman term is 
much more difficult, and we explain the steps in making an estimate on it. 

This term comes from after applying the Kuznetsov trace formula twice and is seen in the 
total geometric side of the trace formula in (15. 6p . 



1 



(2-5) v^E472^m'(^; _ 

^ mV^ pq ^ C1C2 

ci=0(p) x{cx)* y(C2)* 

C2=0{g) 



\ — ^-^)x 



m>l 



Cl 



C2 



C2X + Ciy - kciC2..FM{t) An y/tpqd°m ^tt ^tpg '^i^j 
e{pqt{ — ))^T75-^( : Wi — )^^- 



C1C2 



Cl 



C2 



The functions Fm, F'j^, come from a partition of unity of the approximate functional equation 
of the Rankin-Selberg function. The functions W, V come from our choice of test functions 
in the trace formula. Trivially (12. 5 p has a bound of (pq)'^, we will sketch how to get the 
bound ^ ^ 1-^ — , where 61,62 < 1/4. 

Fix the m-sum for now and look at the ci, C2, and /c-sums. We will now rearrange terms 
following [H]. Naively, the idea is for each integer n finding all ci,C2,x,y,k such that n = 
C2X + ciy — kciC2- Rearranging the sums in this way, we can write x{ci) and y{c2) in terms of 
a paramter r(n). Specifically, for x{ci) we have x = ^^^^^(ci). The advantage of this is that 
the difficult Kloosterman sums are essentially removed. Now again fixing m, we are looking 
at the sum 



(2.6) 

d^mr + -^-^^ x(c^C2)V^(rfc0i/„(^, ^), 

^ — ' ^ — ^ n ^ — ' ^ — ' ^ — ^ vq vq 

n&Z r{n) d\n ci=0(p) C2=-(ci r+X^),C2=0ig) 

(ci,n/d)=l (A,cid)=l 

which is similar to our (]6.10p . Here if„ is nice test function defined in terms of the test 
functions V, W in (16. lip . Denote the Mellin transform of Hn by Then we can write (12.60 
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using Mobius inversion as 



[2.7) 



{pq)' 



rl°mr -L hhUlr 



X 



neZ r(n) 



d\n ci=0(p) 
(ci,n/d)=l 



J(si)=10 J(s2)=10 



{A,cid)=l 

pq 



dci dqf{-cir + 



j-rds,ds2. 



The first step in our estimation is that we can truncate the n-sum with a neghgible error 
(Lemma I6.7p and get the following proposition 

Proposition 2.1 (Step one of estimate). Let p,q be distinct primes and 6,9 > be small 
but fixed. Let Hn{x) be defined as in (16.111) . We have 



(2.8) 



(pq) 



n^O r{n) 



d\n ci=0(p) 
{c\,n/d)=l 



/|ci (A,cid)=l 



J(si)=10 J 



pq 



'(si)=10 J(s2)=10 
(pg)2 27r2 J(si)=2 J(s2)=l/2+<5 J(to)=l/2 

L{x,u))L{tp,si + S2-W 



, / V , ■^—Y'^dsids2 = 

dc' 'dqf{-c,r + ^y 

T{w)T{s2-w) {pqy^+'^ 



L{xi', Si + 2s2-w 

E 

nez 

baedn<^{pq)^ 



r(S2) J9*l+S2-»gS2 

/^(e)x(e) 1 ^(q) /f^)x(&) 



' >- e=l 
Hbaedn{Si, $2) 



E 



d=l a=l 6=1 

1 ,rf°mr + ^r 



r{baedn) 



r 



S2-W 



baedn 



^si+2s2 

)4'{n) dwdsids2. 



We point out, similar to [H], the r-sum on the left hand side of (12. 8 p is almost another new 
Kloosterman sum, the problem is that the inner C2 depends on r. Further, if there were no 
arithmetic conditions on the Ci, C2-sums in (12. 6 p then an easy application of Mellin inversion 
or Poisson summation would suffice to understand how big the Kloosterman-Kloosterman 
term is with respect to p, q. However, we must face these arithmetic difficulties. Following 
|Y] . we use the Mellin transform 



(2.9) 



:i + x) 



1 

27ri 



r{w)r{s — w) 
f(7) 



x-'^dw 



to analytically "separate" the A- sum from the Ci-sum. 
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Now we make a contour shift that passes a pole to give a main term and a remainder term 
from the right hand side of f l2.8p . This gives f l2.8p equahng 



Z-^ gSl+2S2 Z-^ (1^1+32 Z-^ (1^1+32 Z-^ Jj3l+2S2 

>- e=l d=l a=l b=l 



(isi(is2 + {Remainder term fl6.26p }. 



EHbaedn{Sl, S2) \r-^ ~^ 'dP^^ \~Tr^ 

n32 Z^ baedn 

nSZ r(baedn) 
baedn<^{pq)^ 

In (I2.10p we clearly see Dirichlet L-functions that will aid in getting subconvexity in the 
Rankin-Selberg L-function. 

Step two of the estimation will now bring in the m-sum from (12 .Sp to the main term f l2.10p . 
First we define for 3fJ(si) = 2, 3ft(s2) = 1/2 the function 

(2.11) Tm,v{.c):=e[—) I / e(^==— + — 3 ^ 



JO 

00 



hhcy d° k^cx 



00 



e(-== )FM{t)V{^^W{^^)x'^-'y''--'-j:^dxdy. 

y/lihvxy X y t^i'^ 



This is virtually the function if^(si, S2) with an extra exponential sum and with a change 



of variables x — )■ and y — )■ — — 

Proposition 2.2 (Step two of estimate). Withp,q distinct primes, let A = m? — ^j^,m E Z 
and (— ) ^/ie quadratic character with modulus A. VKe have for any e > anc? any A > the 
asymptotic equality 



(2.12) Vp-M-,) ^ ^^,^,,;?!!^){^„„ „^ „j 



m>l 



{pqf ^ 2m 7(,^)=2 i(s2)=i/2 J(w)=eJ-oo PQ ' (pg)^ 

L{S2,X)L{SI,^)L{SI + 2S2 + 1 + W,{^)X) L{S2 + 1 + W,^) ^ £l±£2^ , , , , 

— ^ -, ^ ■ _ B(si+2s2+l+w)v 2 dvdwdsids2 

L{s, + S2,x^) L{sr + 2s2 + l + w,x) L{s2 + 1 + w, ^ ; 

+ 0{ipq)-^). 

Here B{s) is an analytic function for 3?(s) > 1 defined in (16.410 . 

This second estimate comes from the earlier reorganization where we replaced the sum 
x(ci) with the sum r(n). The replacement benefits us in that the m-sum coming from the 
left hand side of (12.121) is a sum of Kloosterman sums that is very similar to Rudinick's thesis 

:^^M'i^)S{d°m, baedn)A{m), 
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with A{x) a nice function. This equation is analogous to our fl6.3ip . Notice also this new 
Kloosterman sum has trivial nebentypus even though we initially started with forms of 
type xip)y4'{'l)- Applying Poisson summation modulo baedn leaves us counting solutions to 
quadratic equations. Such a count is analogous to calculating orbital integrals in the Arthur- 
Selberg trace formula. The beautiful thing now, though it is not essential to the estimate, 
is that these solutions to quadratic equations modulo / depend only the square free part of 
/, and are also multiplicative in /. 

We will ultimately have the contour of the Si-integral on the line 3f?(si) = 1/2. This 
term is analogous to our fl6.42p below. Note from this term we will expect that convexity 
or subconvexity estimates will come into play for Dirichlet L-functions. We also make a 
comment that the m-sum in this case, which is dual to the m-sum from opening the norm 
square of the approximate functional equation, will play the role of the sum of traces of 
conjugacy classes in the Selberg trace formula, see [R]. 

To understand the third step we define 



A := A{x,y,d°,k,li,l2) : = 



H m 



and 



B := B{x,y,k,li,l2) := 



'hkxy 



Then we can rewrite the f-integral in f l2.12p as 



(2.13) 



2 



FM{t)t-^—dt 



2sin(7rw/2) 



Fm\v)v 2 dv 



^0 



2 cos{7nu/2) 



X 



x"*' ^y""^ ^dxdy. 



y 



The steps taken to rewrite the f-integral in this form follow from equations fl6.43p to fl6.53p . 
Let 



(2.14) Da{s,,S2,w):-- 



,A. 
B' 



w/2 



°° r°° 4tt Atc 

/ V{-)W{-)x 
Jo X y 



^(47r J|) - J_^(47r./|) M^ttJ^) + J-^(47r,/| 



2sin(7rM;/2) 



+ 



2 cos(7rw/2) 



x'"^ ^y'"^ ^dxdy 



and 
(2.15) 

C{si, S2,W) 



L{s2, x)L{si, iP) Ljsi + 232 + 1 + (-)x) ^(^2 + l + w,^) 
L{s, + S2,x^) L{si + 2s2 + l + w,x) L{s2 + l + w,{^)i^)) 



B(si+2s2+l+w) 
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Then the right hand side of step two equals 



J(w)=e ^ Uo J L-'-oo 



S]^ + S2 + 2tjj — 1 

2 dv 



Da{si, S2, w)C{si, S2, w)dwdsids2 
+ 0((pg)"^). 



From here a subconvexity estimate is needed for C{si, S2,w) and Bessel function analysis 
is required for D a{si, S2,w). We also point out the interesting fact here that the functions 
Fm , F'j^j, which come from the approximate functional equation are "unhinged" from the 
actual subconvexity calculation. This is good as one can always change the test functions 
involved in the approximate functional equation. We say though, that these integrals of Fm 
and F^^, (via integration by parts) are important in ensuring the convergence of the si,S2 
and ty-integrals. 

Proposition 2.3 (Step three of estimate). For any e > and A and B defined as above, 
we have 

m 

C{l/2 + ih,l/2 + tt2,ty) « {\ti\pf'{\t2\qf'{\ht2y\pqy 
where 61,62 are parameters < 1/4. 

Da (1/2 + ttu 1/2 + it2, Z7) < {tit27-^Y. 

We mention now the remainder term from fl2.10p can be analyzed in the same manner as 
in steps two and three. We unfortunately do not get such a nice product of L-functions as in 
step two, but analogous analysis of that of the main term gets the same bound as the main 
term times an arbitrarily small but fixed power of q. We write these two estimates as our 
final step four of the analysis of the Kloosterman-Kloosterman term. 

Proposition 2.4 (Step four of estimate). For a fixed hut arbitrarily small 5 > 0, we have 
(2.17 ^ —— > -^Pm'{ ){Main term of <^M)} < ^— ; 

(2.18) — > — ttt-Fm'I ){Remainder term of (2.1(J) } < — ; 

m 

(2.19) {Equation (|23])} < ^ 



/pq 

where 61,62 are parameters < 1/4. 

Acknowledgements. I would like to thank Roman Holowinsky for suggesting this prob- 
lem to me. I would also like to thank Matt Young, Paul Nelson, as well as Roman, for 
pointing out some errors in initial drafts of this paper. 
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3. Rankin- Selberg L-function 

Let X s-iid ip be Dirichlet characters modulo prime numbers p and g, respectively, with 
iPil) = 1- Let / G H{p,x) and g G H{q,ip) be newforms associated to GL2 cuspidal 
automorphic representations. The Rankin-Selberg L-function is 



(3.1) L{sJxg):=L{x^,2s)Y, 



af{n)ag{n) 



n=l 



It is known that this L-function has analytic continuation to the complex plane. Following 
[KMV] we can write the Rankin-Selberg L-function as a truncated Dirichlet series using an 
approximate functional equation. Namely, we can write 



L(l/2, / X 9) = E ;v(%Mff (il) + X g) Y: ^£(%Mff.,il; 

n>l n>l 



with |e(/ X g)\ = 1 and defined in |KMV] . The function H (lesp.H') satisfies H(y) <ti y ^, 
for all y4 > 0. As well for y small H [lesp.H') satisfies 

H{y) < {pqYl logy\. 

We will also note that in estimating from line to line {expression}'^ will note bounding by 
an arbitrarily small positive power, and is not necessarily the same e line to line. 

4. Kuznetsov trace formula 

We start by defining the Kuznetsov trace formula. We refer to [Iw] book on it's derivation. 
Let S(To{p), x) be the space of holomorphic cusp forms of weight k for the group To{p). For 
each form G Sk(TQ{p),x), let c„(0) be the n-th Fourier coefficient, then define 



Likewise, for Maass cusp forms we define 

cosh(7rsj 

where has norm one and eigenvalue 1/4 + s"^ with Fourier expansion 

(f){z) = ^p{n)Wsinz). 

n/O 

Here Ws{nz) = 2y/yKs-.i/2{ny)e{x). The continuous spectrum coefficients are defined as 

wt. / , N - 1/2 ntjn) 



r]{l, 1/2 + it) := 27r^+'*cosh(7rt) 



T{l/2 + it)C{l + 2ity 

where Tu{n) = Eab=n(«/^)'*- 

Let V G C^(M+), then the Kuznetsov formula states 

poo 

(4.1) J2h{V,X^)aMai{(i)) + — h{V,t)r]{n,l/2 + zt)riil, 1/2 + it)dt 

471 



/oo °° -| 

h{V,t)tanh{7it)tdt + {k-l)h{V,k) + ^ -S^{l,n, c)V{A7iVhl/c) 



TT 



k>0,k even 



c=0{p) 
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where the sum is over an orthonormal basis for S'fc(ro), A; G 2Z and Maass forms w.r.t. the 
Petersson inner product, and V G C(j"(M — {0}). The Kloosterman sum is 

as well 

V{x)Jx-i{x)x~^dx if A G 2Z; 
JI^V{x)B2ix{x)x~^dx ifAGM-2Z. 



(4.2) h{V,X):- 



Here, B2it{x) = (2 sin(7rit))~^( J_2it(a;) — J2it{x)), where Jfj,{x) is the standard J-Bessel func- 
tion of index /i (See |IK] and |Watj ). 

The delta (main) term of the geometric side of the trace formula has associated to it 



1 if a = 6, 
iia^h. 



5. Including an Amplifier 
We build in an amplifier, and thus our starting point is 

(5.1) h{V,ts)\Y^Ml)? E KWMmil2Jxh)\\ 

where xi are arbitrary complex coefficients. Specifically, |L(l/2, /x/i)p from the approximate 
functional equation above is equal to four sums given by 

(5.2) V h{V,tf)h{WM) y\ , ^,,,, Hi—)H'{—)af{n)afim)ah{n)ahim). 

f<^H(p,x),heH{q,ip) m,n>l ^ ^ J^-n t^-n 

up to a root number e(/ x h). We now introduce a partition of unity for the n- and m-sums 
similar to |KMV] . Let rj{x) be a smooth function which is zero for x < 1/2 and one for x > 1 
such that 



A/>1 

with rjM compactly supported in [M/2,2M] such that x'^7]m{x)^'''' 1 for any i > 0. The 
number of partitions M less than X is O(logX). We use such a partition on the two sums 
n,m, and define 

F{x) := H{x)rj{pqx) = ^ H{x)r]M{pqx) := ^ Fm{x). 

M>1 M>1 

Following the analysis of |KMV] we also have the bound for all i, A > 
(5-3) X —Fm{x) «a, ^(-) 

for any e > 0. The notation for the analogous partition on the m-sum and H' will be 
VM',Flp. Also note by the decay in H (resp. H') in the definition of the approximate 
functional equation we have M < (pg)^'*'^. 
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X 



(5.4) Yl E h{V,t,)h{W,U)\Y,xMl)\' u;^^ 

af{n)af{m)ah{n)ah{m). 



Now opening the square and using Hecke properties, we get 



KL 



h,h<L k\{h,l2) 



Incorporating the above equation into ( 15. 4p for a fixed M, M' from our partition of unity we 
have 



(5.5) Y E E ;^E Jt^^m'I^) 

h,h<L fc|(ii,i2) rfo|!i^ " m>l 



E;^^*^*^~) E h{V,tf)h{W,th)af{n)ah{d°m)ah{n)af{^^m) 

n>l f&H{p,x),h€H{q,i,) 



Let us define 



TT 



/oo 
/i(y,t)tanh(7rt)trft + ^ {k-l)h{V,k) 
-oo 1.^ r, /. . 



k>0,k even 



Then performing Kuznetsov twice and Poisson summation on the ?7,-sum in the 
{Kloosterman-Kloosterman term } we have, as in section 7 of [H], 
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(5.6) Yl E E :^Y.^^'m'(^)'< 

V^Fm(— ii!2™(K)L'„,do^(l^) {delta-delta term} + 

n>l 



-^Fm(—)D iiia^ n/ > ^^^^ — ^ {delta-Kloosterman term} + 

V42^M(-)^n,.o^(W^)( V ' '^"^^ ' ^ V ( V ^ ^ ) {Kloosterman-delta term}+ 



■ ci=0(p) 

1 xd°m y '^'^™ 

C1C2 Ci C2 

ci=0(p) a;(ci)* j/{c2)* 

C2=0{q) 



t^y__oo C1C2 V'^ Ci C2 J 



.,l\l2m 



{Kloosterman-Kloosterman term} 



We will deal with the hardest case first (Kloosterman-Kloosterman term) in Section |6] and 
Section [3, then in Section Owe deal with the delta-delta term. In Section |9] following that, 
we deal with both the delta-Kloosterman term and Kloosterman-delta term. 



6. Kloosterman-Kloosterman term 

We let n = C2X + Ciy — kciC2, and reparametrize the Kloosterman-Kloosterman term of 
(15. 6 p analogous to the equations up to (7.6) of [H], quoting the propositions we need from 
that paper. The only difference between lemmas we need here and that of [H] is that, 
Ci = 0{p) and C2 = 0(g) as there we have only full level. This does not affect the proofs of 
this reparametrization and so we state the propositions we need from jH] . 

Definition 6.1. Let X(ci,C2,n) denote the equivalence classes of pairs {x,y) with x,y E Z 
such that (x,Ci) = 1, (y, C2) = 1, and 

C2X + ciy = n. 

Here we say that {x,y) is equivalent to {x',y') if x = x' (mod Ci) and y = y' (mod C2). Let 
X(ci, C2, n) be a set of representatives for the classes in X(ci, 02, n) . 

Proposition 6.2. Let {x,y) G X(ci,C2,0), then x = —y,Ci = €2- This implies C\ = C2 = 
0{pq). 
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Proposition 6.3. Let {x,y) G X{ci,C2,n) and x E Z be an inverse of x modulo C\ and 
y E Z he an inverse of y modulo c^- Then there exists a 'pair (ri,r2) such that r^r^ = 1 
(mod n) and 

X _ C2 + Ciri _ ci + C2r2 

(6.1) x = , y = 

n n 

The pair (ri,r2) is uniquely determined modulo n by the equivalence class of the pair {x,y), 
and the map from X{ci,C2,n) to the set of pairs (ri,r2) modulo n is injective. 

Definition 6.4. Let C\, Ci he positive integers. Set d = (ci,C2). Assume that d\n. Let 
Y{ci,C2,n) be the set of classes r G (Z/n)* such that 

(a') (ci/rf)r + (c2/rf) = (mod ^) 

(b') (ci/d)r + (c2/rf) ^ (mod f ) zf d'\d and d' < d. 

Proposition 6.5. The map i : {x,y) — )■ ri defines a bijection between X{ci,C2,n) and 
Y{ci,C2,n). 

For proofs of Propositions 16.21 16.31 and 16.51 see |H]. 

Now we break the Kloosterman-Kloosterman term into 2 cases: n = and n ^ 0. 

6.1. [Case n = 0]. This is by far the easier of the two cases. Using Proposition 16.21 above 
for n = 0, we must have ci = C2 = 0{pq) and x = —y. 

(6.2) ^^—^F,,,{—-) ^ ^(^)^(-x)e( -^^)x 

m>l ci=0{qp) 1 x{ciY 



, t^^^ Ci Ci 

Note if d° — = 0, then character sum is zero as -ip ^ x- So we assume for now on 

Now the x-sum above is a Gauss sum with primitive character x4' of conductor pq. In 
order for this sum to be non-zero we must have 

xm{d°-^) xm'a. 

Ci pq 

where m'\m, a\{d° — -^^) , and {m'a,pq) = 1. Further we must have Ci = pqr, where {r,pq) = 1 
otherwise the whole term (16.21) is zero as the support of V, W, and g implies m ^ pq and so 
r must be independent of p,q. 

Then by Chinese remainder theorem the Gauss sum equals 



x{ci)* 



(6.3) > x(a;)V^(-a;)e( ^-^) = > xix)e{ ^ 

Ci — 



p 

x{p)* 



X 



Now to get a non-zero contribution for (16. 2p . 
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With an easy change of variables, fl6.3p equals 



where 



b\ {m,n) 



are Gauss sums, and r = ^. 

Let 
(6.4) 



Z{x) := - 

X 



1 r FM{t)^^AiTVtd^.....^''\/'-^'^ 



1/2 V 



X 



-)W{- 



X 



)dt. 



Denoting the Mellin transform of Z[x) as Z[s), we use Mellin inversion to write the Ci-sum 
in (1S2D as 



(6.5) 



1 _ /a(m(rf°-ii^)) 



ci = Q(pq) 



Cl 



— / Z{s)L{s){^qrds 



for 0" > large, and 
where 

oo 

ci=0(pg) 



/£i(m(^;°-A^)) 



E 



b\m(d°-^) 



1 

6^ 



E 

Cl = l 



The second equality follows from inverting the Ci- and 6-sum. 

In ( 16 -Sp we can shift the contour again to 3f?(s) = 1 bounding the 6-sum by {m{d° — -^^)y 
We have left to estimate 
(6.6) 



Let 



pq Jil) 



1/2 V 



)W^(- 



-)dtx' dx 



ds. 



J{1) J-oo f ' 

Then (16. 6p equals 



X 



-)W{- 



X 



)dtx'' dx 



ds. 



Z'{w)L{l/2 + w-e, x'4'){pqTdw. 



K) 



We can shift this contour to 3fJ(w) = e where e > without hitting a pole. Then we note 



Z'{w) 



pq ' 



1/2 V 



)(itx* '^dx 



X 



'(1) J-oo 

and also by the support of the functions V, W, we get d°k ^ Vhh- 



SUBCONVEXITY FOR THE RANKIN-SELBERG L-FUNCTION IN BOTH LEVELS 



15 



The convexity bound in the level aspect for a Dirichlet L-function, with the level g, is 

L(l/2 + ^^,x)«g'/'+^ 

A sub convexity bound is 

L(l/2 + ^^,x)«g'^+^ 

where 9i < 1/4. Not to discriminate using a convexity or subconvexity bound in this paper, 
we call it a (sub) convexity bound and note (16. 6p is bounded by 

(pg)^-l+^ 

Therefore using the fact that d°q ~ y/hh, with the sum over /i, ^2, we have 

(6.7) ipqr-'^^ Yl TTTW E « (p^f'-'^'L^ 



h,h<L ^ ^' qlihh) 



\x\ 



6.2. [Case n ^ 0]. 



6.2.1. Explanation of this case. Let us summarize what happens in this it is the 
most complicated case in the paper. After using the bijection of [H], we have reparametrized 
the Ci,C2, and k- sums in (15. 6p into a new n-sum. This puts arithmetic conditions on the 
Ci, C2-sums that we remove by standard techniques (Mobius inversion, inverting sums). This 
leaves us with the result of Proposition 16.81 From there we apply Poisson summation in 
the m-sum in Section [6.31 coming from (16. 9p . This lets us exchange Kloosterman sums for 
counting solutions to quadratic equations as in the thesis of Rudnick [R]. This same Poisson 
summation in |R] gives entry to the geometric side of the Selberg trace formula. Now we can 
write the leftover sums coming from the previous Mobius inversion and Poisson summation 
in terms of a product of various L-functions. The sizes of these L-functions will directly 
control how large the final estimation is for (15. ip . The expression in terms of L-functions is 
seen in (16. 42 p . From there we require bounding the archimedean integrals that arise in the 
calculation. After several changes of variables it is evident that the important archimedean 
estimation comes from an unexpected Bessel function that arises. Estimating these integrals, 
including the Bessel function, will only depend on the amplifier parameter L. We see that 
specific dependence only on the amplifier in the analysis of Proposition 16.91 Then we include 
the terms from the amplifier itself and do a final estimation. 

6.2.2. Analysis of n ^ case. Using the bijection we get x = ^^^^^(ci), and x{^) = 
x(c2)x(^) with certain conditions on the Ci,C2-sums which are explicated in section 8 of 

m- 

Let 



/(_ td"m A„ tilil2in 

(6.8, re(-)^„,,^,^),^,WZ5^) * , 

J-oo xy X y ty^ 

then following [H] exactly we have 
(6.9) {Kloo.-Kloo. term} = ^5^5^ -l^n„(^) 

nGZm>l ci=0(p) 

C2=0{q) 



E 



{x,y)eX{ci,C2,n) 



Xix)^p{y)e{— + — )/(n, — , — ; 

Ci C2 pq pq 
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Following the argument of section 8 of [H] closely, the {Kloosterman-Kloosterman term} 
(without the n = term from last section) equals 

(6-10) (^E;^^M^(^) E — 

^■^^^ m>l n^OeZ-r(n) d\n 

EY] xidc2)^p{dci)Hni—,—), 
^-^ pq pq 

ci=0(p) C2=-{cir+A5),C2=0(g) 
{ci,n/d)=l (\,cid)=l 

where 

1 Trim ii hhm 

(6.11) ff„(x, y) := ±e(^ + ^)/(n, x, y). 

xy ny nx 

This is equivalent to fixing n, m, and c? from fl6.10p and looking at 

(6.12) "^(dc) x{dc2)H4^.^). 

ci=0(p) C2=-cir+X^,C2=0{q) 
{ci,n/d)=l (C2,ci)=l 
(X,d)=l 

We apply Mobius inversion on the condition (ci, C2) = 1 to get 

(6.13) Y V^(^ci)$^/i(/) Y x{dfc2)H^{—,'^). 
^-^ ^-^ ^-^ pq pq 

ci=0(p) /|ci C2=-/(cir+A5),/c2=0(9) 

(ci,n/d)=l {A,d)=l 

Using the fact that x is a character modulo p and p|ci, x will only depend on —fXn, 
leaving 

(6.14) 5^ ^(cicO$:M/) E x(-/Agn)g„(^ /^^^~^^" + ^^ 

z — / z — / pq pq 

ci=0(p) /|ci (A,d)=l 

{ci,n/(i)=l 

We will be interchanging many sums and integrals so we require an estimate that allows 
to interchange the n-sum. 

Definition 6.6. A function F ofp,q will be called negligible if for any A > we have 

F{p,q) < (pq)-^. 

Lemma 6.7. The n-sum can be limited to length O^^pq^L'^) for a small but fixed 9 > 0, 
while the complementary sum is negligible. 

Proof. By integration by parts on the integral I(n,x,y) we get 



d°m td°m 



2txJ^ 47r 



X 



X 



,^,^J^^ I dt. 

y y 



Notice by the compact support of V, W, and Fj^, we have d°m ~ pq, t = 0(1), so x is bounded 

d°'^k^ ■ 



and y ~ -Mlh- So we have — <^ — . The functions V, W, Fm are smooth and independent of 
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p, q except for the function Fm{x). One gets from (15. 3p the estimate Fj^{x) <^ {pqY for any 
e > 0. Assuming n ^ [pgYL"^ then by the above analysis (16.151) is bounded by 0{{pqy^^). 
We choose e < ^ so that the integral decays in p and q. Then integration by parts Q times 
gives the bound for (I6.15P of 0{{pq)~^^^'^). For Q large enough and the n-sum in this range, 
(16. Sp is negligible. □ 

We now can limit the n-sum to a length 0((pg)^L^) for a fixed but small ^ > 0. Using the 
lemma and including the n- and c?-sums into (I6.14p we now prove 

Proposition 6.8. Let 6,6 > be small but fixed. We have 
(6.16) 



^^^^ neZ r(n) d\n ci=0{p) /|ci {A,d)=l 



"■¥=0 {ci,n/d)=l 

J(si)=ioi(s2)=io dci dqf{-cir + ^) 

i^{p)x{-q)yjpq, 1 ,3 /" [ f r{w)r{s2-w) {pqy^+''^ 



{pqf '2m' 7(.,)=2 7(s.)=l/2+5 7m=1/2 r(s2) p^i+«2-»g.2 



L{x,w)L{i},Si + S2-w) 
L{xip,Si + 2s2 - w) 



e=l d=l a=l 6=1 



^ Hbaedn{Sl,S2) ^ 1 .d°mr + 

> ^ > e( ; -^-^ — )?/'(n 

neZ r(baedn) 
baedn<t:{pq)^ 



dwdsids2. 



Proof. Incorporating the n and d sum back into (I6.14p . and applying Mellin inversion to the 
Hn function we have 

(6J7) 

^^^'^ neZ r(n) d\n ci=0(p) /|ci (A,d)=l 

"y^O (ci,nM=l 

/" /" Hn{Si,S2){^y'{ — ^y'dSidS2. 

J(sr) J{S2) dci dqf{-cir + ^) 

We need to arithmetically separate the Ci- and A-sum. Following jY], we use the Mellin 
transform 

with ^{w) < 3?(s), to detect the condition 

1 

-cirdqfil + 3^) 
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So the integral can be rewritten as 



■2'Ki' 



T{w)T{s2-w) ^ pq pq 



'dci —Cirdqf 



f'(-^rdwdsids2, 
An 



'{si) J{s2) J(w) 

with 3?(si) = 2, 3f?(s2) = 1/2 + 5, and ^{w) = 1/2 with 5>0 small. 

As the n, ci,A-sums are bounded we can interchange them with the integrals. The next 
goal is to then is detect the many relativly prime conditions with Mobius inversion and 
rewrite the sums in terms of Dirichlet L-functions. We do this in several steps for clarity. 
Fix n, d, and r for now. Apply Mobius inversion to the condition (A, ci) = 1 to get the A-sum 
equal to 

/^(e)x(e) xW 

A=l 



e\d 



Let us collect terms now from fl6.17p . 



(6.19) 



(pq)' 



1 



(si) J{s2) J{w) 



T{w)T{s2 - w) {pqY^+'^ 



H^{s,,S2)^ 1 .d°mr + ^-^r^-^^ ij{d) ^ /i(e)x(e) 



n6Z 



rin) 



n 



d\n 



E 



(ci,n/rf)=l 

Write Ci = pd^^ renaming it C\ and inverting the /-sum with the Ci-sum gives 



dwds\ds2- 



(6.20) 



•27ri' 



Y{w)Y{s2-vS) {pqY 



(si) J {S2) J (w) r(s2) 

1 d°mr + '-^f 



psi+s2-wqS2 



r{n) 



n 



1^' )7Tn)Y^ ^(^) /^(g)x(e) 



d\n 



e\d 



2-^ fsx+2s2-w 2-^ 



/=1 
(/,n/d)=l 



V^(Cl) 

^ ci=l ^1 

(ci,n/d)=l 



dwds\ds2- 



Invert the d- and n-sum to get 

'PQ^{P)X{-(1) f 1 ^3 



(6.21 



[pqf 



(si) J{s2) J{w) 



T{w)T{s2-w) {pqY^+'^ 
pSi+s2-wqS2 



^(^'^)E;^E 



/^(e)x(e) 



E 



^{S2) 
HdniSl, S2) 



d=l 



e\d 



dn<^(pqYL'^ 



E— ' 



.d°mr + '-^r 



dn 



E 



r(dn) 



ci = l ^1 
(ci,n)=l 



dwds\ds2 
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(6.22) 



{pqy 



27^^ J{s^)Jis,)Ji:^) r(s2) r 



S1+S2 — W ^S2 



E 



Hedn{Si, S2) 



e=l d=l neZ 



E— <=( 



d°mr + 



r{edn) 



edn 



-mn) 



2-^ JSl+2S2-W 2-^ 



/=1 

(/,n)=l 



„Sl+S2-«' 

ci = l "^l 
(ci,n)=l 



Finally, we remove the relatively prime conditions on the /- and Ci-sums 



^^-Ki' J^s^)Jis2)JH r(s2) p^i+-2-«;5.2 

Lf W) V ^(^)^(^) ^ /^(") /^^(^)X(^) y^ -^6aedn(gl,g2) 

L e=l d=l a=l 6=1 n&L 

baedn<^{pq)'' L'^ 

r{baedn) f=l ■' ci=l 1 -I 



This can be simplified to 



(6.24) 



v/pgV'(p)x(-g) . 1 . 

L{x,w)L{il),si + S2- 
L{x'ip, si + 2s2-w 

E 




(si) J (S2) J (W) 



T{w)T{s2-w) (pg)"i+^2 

pSl+S2—WqS2 



w 



E 



/^(e)x(e) 

pW+Sl + S2 



1 



e=l 

Hbaedn{si,S2) 



d=l 



a 



S1+S2 



haedn<^{pq)^ L'^ 



E 

r(baedn) 



a=l 6=1 
h^2nt— 



y^/A%(6) 



j^S2 



1 ,d°mr+'-^f^ ^ 
e( ^ 7^ — )ip{n) 



baedn 



dwdsids2- 



□ 
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Remember 3ft(si) = 2, 9?(s2) = 1/2 + 5, and ^{w) = 1/2 with 6 > small. We pick up a 
main term by shifting S2 to the line 3fJ(s2) = 1/2 — 6 with a residue term 



(6.25) 



pqipip)xi-q) 



{pqY 2711 



{si)=2 7(s2)=l/2 P ' 

/^(e)x(e) 1 /i(a) f^'^i^ 



e=l 



E 

baedn-t:(pq)^ L"^ 



d=l 

Hbaedn{Si, S2) 



a=l 



b=l 



r{baedn) 



baedn 



dsids2, 



plus a remainder term 
(6.26) 



'ni^{p)x{-q) , 1 ^3 



(m)^ 



(si)=2 J(s2)=l/2-5 J(«i)=l/2 



L(X,W)/^(V', S1 + S2-W) 



L(xV', Si + 2s2-w 

E 

neZ 
baedn<^{pq)^ L"^ 



e=l 

Hbaedn [Si, S2) 



d=l 



T{w)T{s2-w) {pqY^+'^ 

r(s2) pSl+S2-«'gS2 

^2(6)X(6) 

a 



v^ /^(e)x(e) ^ 1 ^ /i(a) ^-^ fi^jl 

pW+sx+S2 Z^ As\+S2 Z-^ ^si+S2 Z-^ hsi 



E 

r (baedn) 



a=l 6=1 



d°k^ 



baedn 



^si+2s2 

dwdsids2- 



Remark. The "remainder" term is actually bigger then the "main" residual term, but only 
by a fixed but arbitrarily small amount. We expect that this "remainder" term via some 
kind of symmetry (perhaps a functional equation) is smaller than the "main" term. We hope 
that something similar to the analysis in getting the main term of |So] will work in this case. 
Regardless, we focus on the "main" term first. 

6.3. Including the m-sum. Now we want to include the m-sum from fl6.10p into fl6.25p 
and deal with the remainder later in Section [71 The term to consider is 



(6.27) ^t'"y-^' E4?^M-(— )(^)^ 

m>l 



{pqf 

L{x,S2)L{ip,Si] 
Lixi',si + S2) 



{pq) 



S1+S2 



e=l 



d=l 
[Si, S2) 



(si)=2J(s2)=i/2 p''q'' 

f^ \b)x{b) 

^si+2s2 



/^(e)x(e) 1 /^(a) (^ 

Z-^ pSi+2s2 Z-^ flsi+S2 Z-^ ^si+S2 Z-^ hsi 



a=l 



b=l 



n 



S2 



E 

baedn<^(pqf 



Recall Hbaedn{si, S2) depends on m. Specifically, 
(6.28) 



E ^ 

r (baedn) 



.d°mr + ^^f 



baedn 



-)ip{n) 



dsids2- 



Hi,{si,S2) 



~'0 



e{^)FM{t)V{ )W{ 

00 y/Mx 



A /f hhm 



Sl-l„.S2- 



pqy 



)x''-'y 
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ii I 



1'2 



Make a change of variables x VmdPx y _j_ V d°fc^^ gg^- H^{si, S2) equal to 



(6-29) (^:^) 2 ——— / e(-==— + 



J-00 Vhkmxy X y t^l^ 

To simplify notation and to focus on the m-variable define 

xd°^kz yilikf^z 
il2haedny d°'^k%aednoi 

)FMit)V{ )W{ )x'' ^y'' '—j^dxdy. 



00 poo 



(6.30) As,,sM-=z^ / e(^= — ^+ )x 

ylil2baedny d° k-^baednx 



^0 

00 



el , , 

00 Vhhzxy x y 



We interchange the integrals and the e, d, a, 6-sums with the m-sum. This is ok as the 
m-sum is finite. So for the bracketed term of fl6.27p along with Asj,s2(m) (which is just 

Hn{si, S2) relabelled) and the m-sum we have 



(6 31) ^ /^(e)x(e) 1 /i(a) ^ (fc 

e=l d=l a=l 6=1 

—^Z^—m^M'i 2^ e , )As„s2im). 

neZ m>l r(baedn) 

baedn-t:(pq)^ 

Performing Poisson summation in m modulo baedn gives 

1 I1I2 „, ,d°v —mv dv 

(^•22) — —yudo [baedn, m,—-) / F^^i )A,,,,2(i;)e(-— — )^, 

baedn ^-^ d k^ J_nr, PQ baedn v^'^ 

where t'b(n, m, a) := 7^{x : ax^ — mx + b = 0{n)}. 

Following [Rj, Ubi^, m, a) is multiplicative in n, and we can reduce the study of it to prime 
power modulus p^. Suppose first we take n = p, p odd and (a,p) = 1, then completing the 
square 

Ubip, rn, o) = : x'^ = {m2aY — ba{p)}. 
Again as in [R] using Hensel's lemma gives I'bip^, o,) = ^b{p, <i) for k > 1. Then its clear 

(6.33) Ubip, m,a) = l+ [-^- )=[q otherwise. 
Using multiplicativity, we have 

(6.34) z/do(«/3,m,^) = Ud^{fj.''{a/3),m, ^) = Udo{fj.\a),m, ^)udo{ix\(3),m, 



where /i is the Mobius function. The last equality implies we can extend this result to square 



free modulus for u^^ (baedn, m, j^)- 
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6.3.1. Isolating the b, a, e, d, n-sums. In order to isolate these sums, we open up As^^s^[v) and 
focus on the terms containing 6, a, e, and n. Let 



(6.35) T^,i,(c) := ei 



—mv. 



JO 

00 

e( 

-00 



X 



y 



'Iil2vxy' 

Putting (16.351) back into (I6.3ip . using the muhiphcativity of (I6.34p above, and interchang- 
ing the m-sum (after Poisson summation) with the other sums gives 

(6.36) 



pq 



e=l 



d=l 



a=l 



b=l 



baedn<^(pq)^ L'^ 



n 



S2 + 1 



ilj{n)udo{n,m, ^)Tm,v{baedn) 



V 2 dv. 



To completely isolate the b, a, e, d, n-sums in the archimedean functions requires applying 
Mellin inversion to Tm,v{c)- It is easy to check by integration by parts in the t- variable that 

1 



mc)\) 



N 



for any > 0. The variable of the Mellin inversion is w and we integrate on the line 
3f?(w) = e, for e > small. Using the above estimate on Tm,v{c) we move the e, d, a, b, n-sums 
inside of the w-integral to get 



(6.37) 



d°v. 



pq 



T 



(w)=e 



sr^ /^(e)x(e)t^d°(e,m, jk^) ^ z/^o {d, m, j^] 



>- e=l 



d=l 



(^Sl+S2 + 1+* 



^ n{a)udo{a,m,^) ^ ^^{b)x{b)vd° (&, 



a=l 



b=l 



5si+2s2+l+w 



i!{n)udo{n,m, 



baedn<^{pq)^ 



n 



S2 + 1+UI 



dw \ v ^ ^ 



Let A := m — 4-^-, and S := {p prime : p | 2gA}. As the usual notation for primes is 
"p" , we use mathfrak for primes of our Euler product below. Hopefully this does not cause 
too much confusion. Either way, this possible confusion of notation ends at in this section. 
Following for a fixed prime p ^ S*, we have 



(6.38) 



(l + (f))f^_(l + 



^(pKf))^ 



k=l 



(1- 



V'(P)' 



;i-f^) 



Now look at the individual sums in (16.37p . Note with a negligible error we can remove the 
restriction 

baedn <^ [pq^L^. 
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Therefore, the e-sum following what we did for fl6.38p can be written as an Euler product 
equaling 

Here Lp^A(x, s) we do not make specific, but note it is bounded in the range 3f?(s) > 1 by 
{ApqY using fl6.54p . 

Notice the a-sum and d-sum cancel by standard Euler product arguments. 

The n-sum equals 

n ' nw.''^+'+'--). 

The 6-sum is more difficult and its Euler product equals 

(6.41) n ^(p)(i+(A)) n ^^^(^1 + 2.2 + 1 + =: B{s^ + 2.2 + 1 + 

P^-S (1 — pn+2s2+i+™) P^-^ 

Here we do not explicate B^^/^J^s) but analogous analysis to that of the e-sum we have 

n5p,A(s)«(AM)^ 

pe5 

for 3fJ(s) > 1. 

The analytic behavior of the e-sum fl6.39p is the same as 

L(gi + 232 + l + ^^,(^)x) 

up to the factors p & S. Likewise, the n-sum (16.401) is the same as 

and the 6-sum is absolutely convergent for 3fJ(si + 2s2 +w) > 1. We will not address further 
the analytic structure of the 6-sum. Label the 6-sum B{s). 

We consolidate what we have from Proposition 16.81 and the Poisson summation on the 
m-sum getting 

(6.42) 



L{s2,x)L{si,i:)L{si + 2s2 + l + w,{^)x) L{s2 + 1 + w , £i±£2^ , , , , 

— — — r . _ B{si+2s2+l+w)v 2 dvdwdsids2 

L(si + S2,x^) Lis, + 2s2 + l + w,x) L{s2 + l + w,{^)^)) ^ ; 

+ 0{ipq)-^). 

The finite product of Ivp.A(s) for p G S* in the above cases do not affect convergence issues. 
In other words, we can shift the contour of Si,S2 in (16.271) by looking at these quotients of 
L-functions above. 

We aim to bound (I6.42p now. 
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6.4. Bounding the f-integral. The f-integral is 
(6.43) 



d°V 31+32-1 

pq 



OO PCX} 



h 



JO 



'hkhy d° k^hx 



thk 



)FM{t)V{ 



'^'^^^)W{^!^)^x'^-^y'^-Hxdyh^-Hhv 



^lihvxy' ' ' ' a; ' ^ y t^/' 
Change variables h ^ hv, x ^ \/tx and y — )■ y/iy we can write fl6.43p as 
(6.44) 



2 dv. 



FM{t)t^^^dt 



d°V ^ 3i+32+2m-l 

2 dv 



POO POO 4 ^ / PC 



-m. . xd°^k yihhf^. 



hk 



el— — lel , h — o — ^ — )e( , , 

/i VU^hy d°^k^hx' Whkxy 



)K"~'dh ]x''-Y'~ dxdy. 



Note that the f-integral is now bounded by {pqY for any e > with Si = S2 = 1/2. The 
X, y, and t integrals are bounded and so we focus on the /z-integral. 

We completely isolate the test functions Fm, F'j^, by changing v ^ ^ getting (I6.44p 
equaling 



(6.45) 



,pq 31 + S2+2W+1 



S-] -\-sn — l 

FM{t)t — ^ — dt 



Fm'Mv 2 dv 



^0 



An An 



-m xd°^k yjhkf'^ . 



/i Whkhy d°^k^hx 



hk 



'hkxy' 



)h'"''dh ]x''-Y'~ dxdy. 



Remark. So note the choice of Fm, F'j^p from the approximate functional equation in fl5.2p 
is independent of the analysis needed in studying the subconvexity problem for the Rankin- 
Selberg L-function. Therefore if we want, we can bring the M-sum back in and remove the 
partition of unity. 

We will eventually need an estimate for the m-sum to converge. Changing variables from 
h ^ and integrating by parts j-times easily gives the bound for the /i-integral 



(6.46) 
Define 

and 



h 



^hhhy d°^k^hx 
A := A{x,y,d°,k,li,l2) : = 



B := B{x,y,k,li,l2) ■ 
We need to bound the integral, 

(6.47) 



'hhxy' 



xd°^k yikkf/^ 

H -^7—^ m 



hhy ' d°'^k^x 
k 



d°'^km 



'hhxy 



OO A 

e{- + Bh)K"-^dh 
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in terms of pq. We write the /i-integral in terms of Bessel functions by recalling the following 

identities from jlKj . 

(6.48) 

r cos(^(y + -W-'dy = -nMx) sm{ns/2) - nUx) cos{ns/2) = -J-M^-^LM^ 
Jo 2 y 2sm(7rs/2) 

and 
(6.49) 

r sm{% + -W^dy = -7rJ,(x) cos(7rs/2) - 7rY,{x) sm{ns/2) = -n^'^^^ + "^"^^^^ 

Jo ^ y 

So 
(6.50) 



2cos(7rs/2) 



e{fl+Bh)h''"-^dh = -7r(-)™/2 
oh B 



^(47r. /I) - J„^(47r. /I) ^(47r. /|) + J-^(47r. /| 



2sin(7r«;/2) 



2 cos(7rtL'/2) 



Define 
(6.51) 

L{S2, x)L{si, ^) L{s, + 2S2 + 1 + W, (^)x) L{s2 + l + w,^) 

C Si,S2,w := — — — ^ . _ B(Si+2s2+l+w) 

^ ^ Lis, + S2,x^) L{s, + 2s2 + l + w,x) Lis2 + l + w,i^)^lj)) ' ^ 

as well as 

(6.52) Da{si,S2,w) 



n°° 47r 47r 
V{-)W{-) 
X y 

^(47r./|) - J_^(47r7|) ^(47r,/|) + J^^i^TiJ^] 



+ 



2sin(7rw/2) 2 cos(7ri(;/2) 

Then incorporating fl6.5ip and fl6.52p into (16.421) gives 



a;"*' ^y"^ ^dxdy. 



(6.53) 



Ti—) 

m 

J{w)=e " L^O 



(si)=2 J(s2)=l/2 



FM{t)t-^—dt 



-Da (si, S2,w)C{si, S2, w)dwdsids2 + 0{{pq) ^). 

We now make a contour shift on the line 3fJ(si) = 2 to 3?(si) = 1/2. We encounter no poles 
in this shift. 

Now write Si = 1/2 + iti, S2 = 1/2 + it2) and w = iy. We now bound the w-integral. First 
C(l/2 + zti,l/2 + zt2,^2/) « {\ti\pf'{\t2\qf'{\ht2y\pqY, 
where 61,62 are parameters < 1/4. Here we are using 



(6.54) 



[—y<^L{a + tt,x)<^i\t\qy 
1% 



for X a character of modulus q,a > 1 and t G R. We also are using the (sub) convexity bound 

L(l/2 + ^t,x)«(|t|g)'+^ 

with 6 < 1/4 [IK]. 
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Second, we bound 

(6.55) DA{l/2 + iti,l/2 + it2,iy). 
Proposition 6.9. 

(6.56) Da (1/2 + zti, 1/2 + it2, i-f) < iht2-f-y 

Proof. As the x, y integrals are bounded, we first look at the case w = Z7 7^ 0, it is sufficient 
to look at 

(6 57) (A /.Jjni^^ J^ni^) 

^ ■ ^ ^B^ sin(7ri7/2) sinh(7r7/2)' 

Let T(x,y) = Fix w = ^7 7^ 0, then we break the analysis into two cases: I7I > 

T{x,y) and I7I < T{x,y). 

First we study the Bessel asymptotic for T{x, y) smaller than the index. We need the 
behavior for J2i'^{T) for 7 > cT{x,y) and c any positive number. Let z = -^/T^ + 7^, then 
from ([E],pg.87) 

(6.58) J2.,(T) = {2n'/')-'z-'/'e^exp{n^)ei^ _ 2iog(i^))x 

l + ^(i-^) + ^(^-^ + ...) 
2^7 ^8;z 24^3^ (2^7)2^28^2 5762^ ^ 

Incorporating this asymptotic for the appropriate range of integration, the x, y-integral 
can be be bounded up to a constant by 

r.00 ,.00 -I 

51 — 1„,S2 — 1 



/ / V{-)W{-)T[x,yY 
Jo Jo X y 



w^/^^l + {T{x,y)/wy 
The main term is trivially bounded by 



dxdy + remainder terms. 



| 47rVA |iRfa) 



1(7 



1/2 



while the remainder terms are bounded by 



I ^/B I 
|^|l/2+fc 



for k eN. 

Now we study the asymptotic for the argument T{x, y) bigger than the index. For T > I7I 
we use from ( |Wat] pg. 205) 

(6.59) J2.,(r) = -^(P^i(2^7,r)e'^ + 1^2(2^7, T)e-^^), 

where 

— W^{2iy,x) (1 + |a;|)-^cosh(7r?/), 

for i = 1,2, j > 0. 
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So the X, y- integral, in this case, is bounded by 
(6.60) 



27 



V(—)Wi—)Tix,y) 

Jo ^ y 



47-1/2 



Wi{2i-f, T{x, y))e'^(^'y^ + W2i2i-f, T{x, y))e-^^(^'J^) 



sinh(7r7) 



1/4- 



Without loss of generality this argument works for the other Bessel functions divided by 
the hyperbolic functions in (16 .52 p . 
Now for the case w = 0, 



lim 

10— >0 



J^(47rT) - J_^(47rT) ^ J^{At^T) + J_^(47rr) 



ro(47rT) 



+ Jo(47rT). 



2sin(7rw/2) 2cos(7rw/2) 
Here we use the continuity of the index of the Bessel function ( |GR] 8.487) and the definition 

Jw{x) cos(7rw) - J-yj{x) 

lyj^X) : -. r . 

sin(7rw) 

Assume \T{x^y) \ is larger than an absolute positive constant C. Then for the second term 
Jo(47rT) we can use the same analysis as is used to get (I6.60p . For the first term we can use 
the asymptotic 



Y,{z) = J^sin{z-l) + 0{\z\-^] 

V T^Z 4 



[GRj . Similar to (I6.60p we can bound the x, y- integral up to an absolute constant by 



1 « 1 



^/T(x,y) Ve- 
il \T{x, y) \ is smaller than C, then we use the first term of the power series expansion for 
Jo(47rT) and the asymptotic formula |GR] 

Fo(^) = -[ln(;^/2)+7], 

TT 

with 7 Euler's constant. The integral in this case can be bounded by ln(27rT) <^ . 
If T(x, y) = 0, the x, y integrals converge and are independent of the parameters p, q. 



□ 



6.5. Bounding the si, S2, w-integrals. Recall we are trying to bound 



(6.61) 



pq 



{pqr^J_ik.n+^^ 



{w)=€ 



FM{t)t^^dt 



L "'0 



-Da (si, S2, w)C(si, S2, w)dwdsids2 
+ 0{{pq)-^). 

By the previous section we have a bound for the functions Da(si,S2,w) and C(si, S2,w). 

B 

We must now ensure the Si, S2, and w-integrals converge. 
Using integration by parts we have 



Si +S9+2ii? — 1 



Si + S2|)\M 



dt « C-^^f^) 

^ 1+ w ' 



28 



P. EDWARD HERMAN 



for any M > 0. We also have 



'dt < 



;i + |si + s2i) 



N ■ 



for any N > 0. Clearly, if we choose > M + 2 > 2 all three of the integrals converge. 

Remember the m-sum converges by the estimate (16.461) . If we include the h, /2-sums into 
(I6.53p . we have 



(6.62) 

^(p)x(-g) 

pq 



9lT 



li,h<L k\Ui.h) ^o^hh 



(w)=e 



FM{t)t^^dt 



L "'0 



(si)=l/2 J(S2)=1/ 



-1/2 p'^q"' 



D{si, S2, w)C{si, S2, w)dwdsids2 + 0{{pq) ^) 

7° 



< — ^— E ^hw^il'^ E ^ E xTT^ ^ 



pq 



h,h<L 



k\{h,l2) rfolk^ 



with 01,^2 < 1/4. 



7. Remainder term from (16.261) 

In turns out in this case the "remainder" term is bigger than the residue term with respect 
io pq (but only by a fixed but arbitrary amount), however the analysis is essentially the same. 
We hope that there is a hidden symmetry similar to |So] so that this "remainder" term cancels 
via some kind of functional equation and leaves the beautiful main term (I6.53p . 

We study 



(7.1) 



»(p)x(-g), 1 ,3 /• f f r(a)r(s2-a) (m)'^+' 



27rZ J{si)=2 J{s2)=l/2-5 J(a)=l 



{pqf '2m' Jis,)=2J(s2)=l/2-sJ(a)=l/2 r(s2) p^i+^2-agS 



L(x, Q!)I/(^/', Si + S2- a 



L{xip, si + 2s2 - a) 



/^(e)x(e) 1 f^ja) [AW^) 

e=l d=l a=l 6=1 

E ijjyujHbaednjSi, S2) ST^ 



i^{n)Hbaedn{Si,S2) ^ 1 _ rfW + . 



nSZ r(baedn) 
baedn<t:{pq)^ 



dwdsids2. 



As in the main term we will execute Poisson summation in the m-sum modulo baedn; the 
analogous interchanges of sums and integrals is identical. 

1 1 /"°° , d°v —mv dv 

m r{baedn) 

{r:'j^2^-mr+d°=0(baedn)} 
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Applying the same steps up to fl6.42p gives 



(7.3) 



L{SI + 2S2 - a, X-ip) J(n,)=e J-oo PQ 



e=l d=l a=l 



f^\b)x{b) ^(n) , 1 

Z-^ Usi+2s2+l+w Z-^ ^a+l+ui V Z-^ 



^si+2s2+l+w / ^ ij^a+l+ui V / J ^S2+w—ci I 

= 1 nSZ r{baedn) 

n^O , , 2 

{r;-^li^^ — mr+d°=0(baedn)} 



V 2 dvdwdsids2+0{{pq) 



We shift the (si) = 2 to the hne 1/2 + 5. There are no poles encountered in this contour 
shift. 

The same archimedean analysis as is done in Section 16.41 simplifies this term to 



(7.4) 

VpqHp)xi-q)^, I ,3 f f Jik,ii±i2 f r{a)r{s2-a){pqy'{pqY' 



~~ ^TTi J (s-,)=l/2+S J (s2)=l/2-S 1^ J{a)=ll 



{Pq? ™ 27r2^ J(s,)=l/2+5J{s2)=l/2-5 J{a)=l/2 ^{^2) p^i+^^-a^s 



L(a,x)L(si + S2 - tt,V^) 



L{si + 2s2- a,x^) J(w)-- 



I n(^ „,Jv^ ^(e)x(e) 1 M 

^{Sl, S2, W) c«+si+2s2+l+i« ^si+S2+l+w ^si 
J(w)=e L „_i j_i " 1 " 



e=l " d=l a=l 



Z-^ fjSi+2s2+l+w Z-^ j^a+l+w \ Z-^ J.S2+W-01' 

6=1 neZ r{baedn) 

™7^0 , , 2 

{r:-^^ — mr+d°=0{baedn)} 



dwdsids2 + 0{{pq) '^), 



with again D{si, S2,w) defined as in (16.521) . 

The r-sum has at most two terms in it following (I6.33p . so it is bounded by 2{haednY . The 
entire sum inside the brackets of (17.41) is absolutely convergent and is bounded (in terms of 
p,q) by (pg)^ Using (I6.54p gives 

^7 5^ L{a,x)L{si + S2-a,ij) ^ e,+e e2+e 

L{si + 2s2~a,x^) 

with 61, 62 < 1/4. The same analysis is done on D{si, S2, w) as in the previous section. The 
only difference between the analysis of this section and the previous section is the a-integral. 
We can use an application of Stirling's approximation to the ratio of Gamma functions as 
in |Y] to get 

^'°'';'V°' «ii+Mi-f)r'^^ 

i (■S2) 12 

where a = 1/2 + it and S2 = 1/2 — S + it2 with t,t2 large. A standard integration by parts 
argument in the a-integral ensures that it converges. 

So analogously to getting the bound (16.621) . (I7.4p is bounded by 
(7.6) 

p0iqe2+5^pqy^ 1-5/2,1-5/2 V- 1 V- p^q'^^' {pqY 

Xi^Xi^L^ L2 p/2-<5/2 To3/4 ^ 

h,h<L k\(h,l2) 



l-25+e| I I |2 



X\ 



2 
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with 61,62 < 1/4. 

Combining fl6.62p and f l7.4p we get the Kloosterman-Kloosterman term with n 7^ is 
bounded by 

(7.7) ^-^^L'^'MUl + q'). 



8. Delta- Delta term 
The term from (15.61) that we now study is 
(8.1) 

h,l2<L q\{h,l2)d°\'^ 

Now this term is non-zero only if n = and n = d°m, which imphes A//1/2 = So we 



can bound (18. ip by 

, v-^ v-^ 1 1 ^; ,d°m,^ ,d°m. 

(8.2) 5: E ^E-'-M'i— )■ 

7^2=9^° 

Using a standard Melhn inversion we can reduce to 

^•^'^ h,h<L ^ ^ "^^ k\{hM) 

for any M > 0. 

An easy estimate for this last line is (pg)'^L'^| |a;| I2. 

9. Delta-Kloosterman term 

It is obvious the analysis for this term will suffice for the Kloosterman-delta term. The 
term is 

(9.1) Vi2 E E 7T7^E47^^M'(^)E^^^^(-)^niii2;i(^)x 

S-,p{n,d°m,C2) ATT\/n(Fm, 

C2=0((?) 

Similar to the previous section, the only non-zero term is when n = which then reduces 
dnU) to 

(9-2) E 7n% E ^ E E-^M^(^)i^-(^)x 



C2 Co 

C2=0(g) 



This calculation is very similar to 
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We isolate the m-sum to get 



'Fm{^^) 2^ i^{y)e{^ )W{ ^- ). 



m pq pq T~ C2 C2 

This is very similar to f l6.3ip above. Performing Poisson summation on the m-sum modulo 
C2, and including the C2-sum we get 



1 /-CJO 70J. tit2r ^ 

(9-3) E -E E ^(^) / nA-)FM{^)w{- 



C2=0{q) m y{c2Y 



C2 C2 t 



Let us change variables t — )■ giving 



kt 

d°kt s/hht 



Ant. , —mat .dt 

)e( -) — . 

C2 vhkc2 t 



(9.4) E -E E ^(^) / i^M'(^)^M(^)iv( 

ili^j^2_„y+do=0(c2) 

We bound the y-sum by 2 as in Section 16.31 and define 

J-oo pq pq X X t 

Then interchanging the m-sum and C2-sum with another application of Mellin inversion, (19. Sp 
is bounded by 

(9-5) 2E E -Tm{c2) = 2J2{^) [ TUs)^-^^ds = 



where 5 > 0. The main term is 



2E- 



g Jo i-oo M ^ X t a; 

As Fm, FIj,,W are compactly supported y/hh ~ gc^", the t-integral and x-integral is of 
bounded size (independent of p,q due to the multiplicative Haar measures). The m-sum 
converges by an integration by parts argument. Also the m-sum can be truncated at length 
0{pq), with a remainder term having a bound 0{{pq)~^), for any M G N. Integrate by 
parts once in the t-integral to gain a term of size d° ~ v^Si^ ^jj^g gj^g of the integral is of 
size |f , and so bounding the m trivially gets the whole main term is of size 0{p'^q'^~^). The 
remainder term by an analogous argument gives 0{p'^q'^~^) also. 

Our leftover analysis is then 

(9.6) fq'-' Yl ihhV^xi.xi, q<^PY-'L%A\l 

h,l2<L glihM) 

Now one can apply the same argument as we did for the delta-Kloosterman term to the 
Kloosterman-delta term, and get analogous bound 0{q'^p'^~^), and bounding the li,l2 sum 
gives g''p^~"'"L^| I2. 
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10. POSITIVITY OF TEST FUNCTIONS ON SPECTRAL SIDE OF TRACE FORMULA 
In order to go from estimating 

(10.1) J2 77^M^'^/)lE^'^/(0l' E jj^MW,t,)\L{l/2Jxh)\' 
to a single term 

(10.2) (/, f){h, h) ' 1^ ^A/(0 n^(l/2, / X h)\\ 

we need careful choices of our test functions V, W. We use the following proposition of |Venklj : 

Proposition 10.1. Given jo £ e > and an integer N > 0, there is a V of compact 
support so that h(y, tj) = 1, and for all j' ^ jo, h{V, tj/) -C e(l + |tj/|)~^, and for all k odd, 
h{V, k) < tk~^. 

Given /cq, e > and an integer N > 0, there is a V of compact support so that h{V, k^) = 
1, h{V, k) < ek-^ for k odd k ^ ko, and h{V, t) < (1 + \t\)-^ for all M. 

Using the above proposition we can choose V, W to isolate spectral parameters {t p , tho } 
with Ny, Nw large enough and e = ey = small enough so that the sum 

(10.3) 77^W^/)lE^'^/(0l' E -7^MW,tr.)\L{l/2Jxh)\^<e. 
With this choice of V, W fll0.2p we have the inequality 



(10-4) ^j^J^^^\Y.^Ml)?\L{l/2J° X « 



KL 



11. Putting it all together 

Using the estimates from Sections El [TJ [H and |9l and letting = = 6*2 in (17. 7p as well 
as ( 16. 7p . we get (putting explicitly in the orthonormal basis built into the trace formula) 

(11.1) E 77^M^,t/)iE^'^/(or E jj^MWM)\L{i/2j X h)\' <^ 

1 1^112 ^(p9)*^-t'H'^^lta-delta term} 
+ p'^g'^~^L'^{delta-Kloosterman term} + g'^p'^~^L'^{Kloosterman-delta term} 

+ {pqY'^^'L' + ipqf''^^'^'^'{l + g'^)L^+'{Kloosterman-Kloosterman term}.^ 

Remark. (1) If we take L = 1, the trivial amplifier, we obtain the expected Lindelof on 
average bound even with 6 = 1/4. 
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(2) Remember the "abnormal" term {1 + q^) in the Kloosterman Kloosterman estimate 
comes from a contour shift to get the "main" term and "remainder" term in fl6.25p 
and (16 .26 p . The Kloosterman-Kloosterman bounds above in (lll.ip come from these 
respective terms' analysis in (I6.62p and (17. 6p . We expect with more refined analysis 
and exploiting some kind of symmetry or functional equation, similar to what is done 
in [So], the "remainder" term is much smaller than the "main" term. 

(3) The term {1 + q^) being not symmetric in p and q is an artifact of that we are 
only amplifying the forms of level p in (lll.ip . If we put an amplifier on both forms 
/ G H{p, x) and h G H{q, ip) we would again expect a symmetry in the estimate. 

(4) Note in this paper, we do not apply any Ramanujan bounds towards the Fourier 
coefficients as all the analysis is done on the geometric side of the trace formula. The 
only crucial ingredient is the subconvexity bounds for GLi L-functions. 

As mentioned, it is implied in the trace formulas used above that the /- and /i-sums are 
orthonormal with respect to the Petersson inner product. This inner product also depends 
on the level p and q. Using that (/, /) ^ p{\ogp)^ and {h^h) <^ g(logg)^, and a choice of 
y, W as in section [10] to get the inequality (110. 4p we get 

I ^i^P (0 n^(l/2, r X |2 < I |x| |2 ({pqf+^U + (pg)^+i/2+.^^ ^ q')L^+^+ 

1<L ^ 

{pqf^'L' + p^^'q'L' + q^+YL'^ ■ 

We now choose the coefficients as in |KM Vj . 



Afo(/) if / is prime < L^/^ 
1^11.2) xi := — 1 if Hs a square of a prime < L^/^ 

otherwise. 

Now using the fact that 

|^xAr(/)r»(M)-^^ 

1<L 

and 

\\x\\l<^L'/'+% 

we have 

(11.3) |L(rx/iM/2)| « (pqy(^(pq)lL~'/^ + (l + q^)y\pq)l^'.L-^+^^^ + P'^ 
We can simplify it to 

(11.4) |L(r X k', 1/2)1 « („).( (w)' + (P^)'^;^+p'^' + 9'^' ^ ,1 ^ ,Y'(pS-iLiJ , 

If we choose L = {pq)^~^ then (111.31) is bounded by 

{pqr{{pq)'^~"' + {pqf'^'^ + p^/^+lgf-i/^ + g^/s+fpf-i/s + (i + g^)(pg)f + !^. 
This completes Theorem 11.11 
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